Workshop on Computational Personality Recognition: Shared Task

In the Workshop on Computational Personality Recognition (Shared Task), we released two datasets, varying in size and genre, annotated with gold standard personality labels. This allowed participants to evaluate features and learning techniques, and even to compare the performances of their systems for personality recognition on a common benchmark. We had 8 participants to the task. In this paper we discuss the results and compare them to previous literature. Introduction and Background Personality Recognition (see Mairesse et Al. 2007) consists of the automatic classification of authors’ personality traits, that can be compared against gold standard labels, obtained by means of personality tests. The Big5 test (Costa & MacCrae 1985, Goldberg et al. 2006) is the most popular personality test, and has become a standard over the years. It describes personality along five traits formalized as bipolar scales, namely: 1) Extraversion (x) (sociable vs shy) 2) Neuroticism (n) (neurotic vs calm) 3) Agreeableness (a) (friendly vs uncooperative) 4) Conscientiousness (c) (organized vs careless) 5) Openness (o) (insightful vs unimaginative). In recent years the interest of the scientific community in personality recognition has grown very fast. The first pioneering works by Argamon et al 2005, Oberlander & Nowson 2006 and the seminal paper by Mairesse et al. 2007, applied personality recognition to long texts, such as short essays or blog posts. The current challenges are instead related to the extraction of personality from mobile social networks (Staiano et al 2012), from social network sites (see Quercia et al. 2011, Golbeck et al 2011, Bachrach et al. 2012, Kosinski et al. 2013) and from languages different from English (Kermanidis 2012, Bai et al 2012). There are also many other applications that can take advantage of personality recognition, including social network analysis (Celli & Rossi 2012), recommendation systems (Roshchina et Al. 2011), deception detection (Enos et Al. 2006), authorship attribution (Luyckx & Daelemans 2008), sentiment analysis/opinion mining (Golbeck & Hansen 2011), and others. Copyright c © 2013, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. In the Workshop on Computational Personality Recognition (Shared Task), we invited contributions from researchers or teams working in these areas or other related fields. Despite a growing number of works in personality recognition, it is still difficult to gauge their performance and quality, due to the fact that almost all the scholars working in the field run their experiments on very different datasets, and use very different evaluation procedures (Celli 2013). These problems are exacerbated by the fact that producing gold standard data for personality recognition is difficult and costly. In 2012 there has been a competition on personality prediction from Twitter streaming data1 with about 90 teams participating, thus showing the great interest of the industry and the research community about this field. The Workshop on Computational Personality Recognition (Shared Task) is different from a simple competition, because we do not want to focus just on systems’ performances, but rather we would like to provide a benchmark for discovering which feature sets, resources, and learning techniques are useful in the extraction of personality from text and from social network data. We released two datasets, different in size and domain, annotated with gold standard personality labels. This allowed participants to compare the performance of their personality recognition systems on a common benchmark, or to exploit personality labels for other related tasks, such as social network analysis. In this paper we summarize the results of the Workshop on Computational Personality Recognition (Shared Task), discussing challenges and possible future directions. The paper is structured as follows: in the next section we provide an overview of previous work on personality recognition. Then in the following sections we present the datasets and the shared task, we report and discuss the results, and finally we draw some conclusions.